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NASA STI Program ... in Profile 


Since its founding, NASA has been dedicated to 
the advancement of aeronautics and space science. 

The NASA scientific and technical information (STI) 
program plays a key part in helping NASA maintain 
this important role. 

The NASA STI program operates under the 
auspices of the Agency Chief Information Officer. It 
collects, organizes, provides for archiving, and 
disseminates NASA’s STI. The NASA STI program 
provides access to the NASA Aeronautics and Space 
Database and its public interface, the NASA Technical 
Report Server, thus providing one of the largest 
collections of aeronautical and space science STI in 
the world. Results are published in both non-NASA 
channels and by NASA in the NASA STI Report 
Series, which includes the following report types: 

• TECHNICAL PUBLICATION. Reports of 
completed research or a major significant phase 
of research that present the results of NASA 
programs and include extensive data or 
theoretical analysis. Includes compilations of 
significant scientific and technical data and 
infonnation deemed to be of continuing 
reference value. NASA counterpart of peer- 
reviewed formal professional papers, but having 
less stringent limitations on manuscript length 
and extent of graphic presentations. 

• TECHNICAL MEMORANDUM. Scientific 
and technical findings that are preliminary or of 
specialized interest, e.g., quick release reports, 
working papers, and bibliographies that contain 
minimal annotation. Does not contain extensive 
analysis. 

• CONTRACTOR REPORT. Scientific and 
technical findings by NASA-sponsored 
contractors and grantees. 


• CONFERENCE PUBLICATION. Collected 
papers from scientific and technical 
conferences, symposia, seminars, or other 
meetings sponsored or co-sponsored by NASA. 

• SPECIAL PUBLICATION. Scientific, 
technical, or historical information from NASA 
programs, projects, and missions, often 
concerned with subjects having substantial 
public interest. 

• TECHNICAL TRANSLATION. English- 
language translations of foreign scientific and 
technical material pertinent to NASA’s mission. 

Specialized services also include creating custom 

thesauri, building customized databases, and 

organizing and publishing research results. 

For more information about the NASA STI 

program, see the following: 

• Access the NASA STI program home page at 
h ttp://www. sti. nasa. gov 

• E-mail your question via the Internet to 
help@sti.nasa.gov 

• Fax your question to the NASA STI Help Desk 
at 443-757-5803 

• Phone the NASA STI Help Desk at 
443-757-5802 

• Write to: 

NASA STI Help Desk 

NASA Center for AeroSpace Information 

7115 Standard Drive 

Hanover, MD 21076-1320 
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1.0 

Authorization 




The Office of Chief Engineer requested that the NASA Engineering and Safety Center (NESC) 
lead a small group to develop a proposal for a common taxonomy to be used by all NASA 
projects in the classifying of nonconformances, anomalies, and problems. The intent was to 
determine what information is required to be collected and maintained in order to facilitate 
trending and root cause analyses in addition to assisting individual problem resolution. This task 
was within the scope of NESC’ s charter, where NESC is tasked with performing “independent 
safety and engineering trend analyses and technical risk assessments utilizing program and 
discipline data sources and state-of-the-art tools and techniques while looking for trends across 
and within programs.” 
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3.0 List of Team Members 

NASA personnel with diverse experience in both human space flight and robotic missions were 
recruited to participate in this activity. Team members had expertise in knowledge management 
systems, anomaly resolution, trending, current problem reporting systems, and taxonomy 
development. Managers at the various centers endorsed this work by funding their employees’ 
participation. The team consisted of: 


Team Members 


Name 

Center Affiliation 

Vickie Parsons 

NESC, LaRC 

Robert Beil 

NESC, KSC 

Tina Panontin 

ARC 

Roxana Wales 

ARC 

Michael Rackley 

GSFC 

James Milne 

GSFC 

Tim Barth 

KSC 

John McPherson 

MSFC 

Mark Terrone 

NESC, KSC 

Jayne Dutra 

JPL 

Larry Shaw 

JSC 

Support 

Elizabeth Holthofer 

ViGYAN, Inc., LaRC 
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4.0 Executive Summary 

The purpose of the Taxonomy Working Group was to develop a proposal for a common 
taxonomy to be used by all NASA projects in the classifying of nonconformances, anomalies, 
and problems. Specifically, the group developed a recommended list of data elements along with 
general suggestions for the development of a problem reporting system to better serve NASA’s 
need for managing, reporting, and trending project aberrant events. 

The definitions, suggested values, and prescriptions for various fields provided in this report and 
the appendices are guidelines for future (and existing) NASA projects. The authors recognize 
that individual projects have needs that may require a finer dissection of the data, while others 
may need less information to adequately manage their nonconformances, anomalies, and 
problems. The bottom line is that there is a critical need for projects to capture information on 
aberrant events in order to determine the causes and prevent future occurrences. Where an 
individual project captures the data in a different format, the relevant data needs to be translated 
into the shared data elements and provided to a common source so that trending across projects 
may be accomplished by independent organizations such as the NESC. Submittal of this report 
to NESC and Office of Chief Engineer management concludes the work of the Taxonomy 
Working Group and this team will be dissolved. Finally, it is advisable to have an expert panel 
‘scrub’ the taxonomy of existing field codes to ensure they are accurate, complete, and 
unambiguous. This panel should include experts in taxonomy development as well as experts in 
problem reporting for major NASA programs. They should also ensure that Cause Codes refer 
only to causes, Defect Codes only to defects, etc. 
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5.0 A/I/C Plan 

Not applicable. 
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6.0 Description of the Problem and Proposed Solutions 

Purpose 

The purpose of the Taxonomy Working Group was to develop a proposal for a common 
taxonomy to be used by all NASA projects in the classifying of nonconformances, anomalies, 
and problems. Specifically, the group developed a recommended list of data elements along with 
general suggestions for the development of a problem reporting system to better serve NASA’s 
need for managing, reporting, and trending project aberrant events. Since a taxonomy is a 
controlled term list, not a data architecture for a particular system, the intent was not to design a 
problem reporting system. However, the recommendations within this document may serve as a 
partial guide to system developers in the future. 

Proposed Solution 

Appendix A provides details of the data elements recommended for any NASA project to collect 
and maintain for nonconformances, anomalies, and problems. The generic formats for each data 
element and suggested taxonomies or potential values are also included in Appendix A. This 
complete set of data elements should provide enough information to facilitate the root cause and 
trend analysis required of the individual projects by NPR 7120.5C. Data elements marked with 
an asterisk in the share column represent the minimum set of data elements that all projects must 
make available through a common user interface to organizations, such as NESC, tasked with 
performing independent trending across projects. With the understanding that some projects 
currently have systems that do not contain all these asterisk items, some reduction in this 
requirement is identified within Appendix A by indicating which of those data elements would 
only be required of new projects or as applicable (marked as ‘New’ in the shared field). 
Additionally, given the differences between human spaceflight and robotic mission life cycles 
and post launch activities, some reduction in data elements may be further requested. Appendix 
A is the starting point from which individual programs and projects should develop their data 
requirements and problem reporting systems. Where pick lists or taxonomies are provided for 
individual data elements, these are meant to be suggestions and may not be comprehensive as a 
project determines the necessary values when their actual system is being developed. However, 
certain coding schemes (i.e., criticality codes) should be consistent from one program to another 
in order to facilitate comprehensive NASA trending. Also, the Taxonomy Working Group 
identified individual data elements within Appendix A according to when the data would most 
likely be available for entry into a problem reporting system (initiation, analysis, or closeout). 
Appendix B provides a summary of the characteristics of a good taxonomy for project reference 
when further developing their individual systems. 
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7.0 Data Analysis (Refer to Appendices for additional information). 
Not applicable 
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8.0 Recommendations 

Appendix A provides details of the data elements recommended for any NASA project to collect 
and maintain for nonconformances, anomalies, and problems. The generic formats for each data 
element and suggested taxonomies or potential values are also included in Appendix A. This 
complete set of data elements should provide enough information to facilitate the root cause and 
trend analysis required of the individual projects by NPR 7120.5C. Data elements marked with 
an asterisk in the share column represent the minimum set of data elements that all projects must 
make available through a common user interface to organizations, such as NESC, tasked with 
performing independent trending across projects. With the understanding that some projects 
currently have systems that do not contain all these asterisk items, some reduction in this 
requirement is identified within Appendix A by indicating which of those data elements would 
only be required of new projects or as applicable (marked as ‘New’ in the shared field). 
Additionally, given the differences between human spaceflight and robotic mission life cycles 
and post launch activities, some reduction in data elements may be further requested. Appendix 
A is the starting point from which individual programs and projects should develop their data 
requirements and problem reporting systems. Where pick lists or taxonomies are provided for 
individual data elements, these are meant to be suggestions and may not be comprehensive as a 
project determines the necessary values when their actual system is being developed. However, 
certain coding schemes (i.e., criticality codes) should be consistent from one program to another 
in order to facilitate comprehensive NASA trending. Also, the Taxonomy Working Group 
identified individual data elements within Appendix A according to when the data would most 
likely be available for entry into a problem reporting system (initiation, analysis, or closeout). 
Appendix B provides a summary of the characteristics of a good taxonomy for project reference 
when further developing their individual systems. 

In addition to the main deliverables provided in Appendices A and B, the Taxonomy Working 
Group makes the following recommendations as projects begin developing their problem 
reporting systems. 

Recommendations 

R-l Projects should require that every contractor/vendor/civil servant enter ALL anomalies 
into a common system for the project rather than have different systems for different 
levels of aberrant events. Maintaining all project data on nonconformances, anomalies, 
and problems within one system will facilitate trending and early identification of 
potential problems. Additionally, it allows universal access to the data ensuring 
commonality. 

R-2 Problem reporting systems should be seamlessly integrated/linked with other databases 
such as Failure Mode and Effects Analysis (FMEA), critical items list (CIL), Hazard, 
Mishap/ Incident Reporting Information System (IRIS), hardware, Govemment/Industry 



NASA Engineering and Safety Center 
Working Group Report 

Document #: 

RP-06-11 

Version: 

2.0 

Title: 

Taxonomy Working Group 

Page #: 

11 of 51 


Data Exchange Program (GIDEP), logistics systems, and action item tracking systems for 
the sharing of pertinent information. 

R-3 NASA should utilize their contract negotiations to require standardization across vendors 
for part numbers and naming conventions. 

R-4 NASA should consider a database of common hardware to include information on life 
cycles and use times. 

Best Practices 

1 Problem reporting systems should be designed to generate actions based on certain values 
in critical fields and populate a standard action item tracking system automatically. 

2 Problem reporting systems should insure that all related data is visible and usable with no 
hidden data. 

3 Problem reporting systems should be designed to allow searches for specific values 
within fields. 

4 Problem reporting systems should be designed to automatically complete related fields 
where possible rather than require manual entry. For example, where the criticality code 
is known from other data systems, the problem reporting system should import it rather 
than requiring the user to create it. Also, it is recommended that for information about 
NASA employees and contractors, problem reporting systems use the POPS2: People, 
Organizations, Projects, Skills schema that is incorporated into the National Institute for 
Science Education (NISE) metadata framework and implemented into the Lightweight 
Directory Access Protocol (LDAP) Directory. The schema includes information about 
Competency, Location, Title, Contact Information, Organization and Employee Number. 
The unifonn resource locator (URL) with specific schema information is available with a 
password as a Raw Data File (RDF) file at this location: 
http://lurch.hq.nasa.gOv/2005/l 1/21/pops.owl 

5 Problem reporting systems should employ pick lists and eliminate the use of meaningless 
data codes. 

6 Wherever appropriate, pick list fields should allow multiple choices rather than force the 
user to determine one option. 

7 Wherever appropriate, pick lists should include the option to enter something under an 
“other” category in the event that the pick list is not comprehensive. 
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8 Problem reporting systems should include prompts, explanations, and examples within 
the free form text fields to guide the user towards a good entry. Suggestions for several 
key fields are provided in Appendix C. 

9 Problem reporting systems should include the ability for updates to individual fields as 
more information is obtained. For example, the problem description may require several 
updates as the investigation proceeds and more data is gathered. The system should 
automatically maintain an archival record and update log as new entries are made and/or 
updated. Problem reporting systems should be designed to keep configuration control of 
individual problem records and easily identify when the record was last updated and by 
whom. 

10 Problem reporting systems should include logic to guide the user in data entry and 
preclude entry of impossible combinations. The underlying databases beneath problem 
reporting systems, through what are called ‘business rules’ in the information technology 
(IT) community, should be burdened with the task of enforcing that all such fields are 
filled in at the appropriate stage in the problem report life-cycle, as required. When 
‘other’ is selected for a given code, the database should then prompt for a textual 
description of the actual cause, defect, etc. 

11 Problem reporting systems should have embedded help information and tutorials to 
enhance usability for reporters, analysts, and managers. 

12 Problem reporting systems should include the capability to attach related documents, 
pictures, figures, etc. 
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9.0 Lessons Learned 

Not applicable. 
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10.0 Definition of Terms 

Clarification Definitions 


It was evident from the discussions within the Taxonomy Working Group that NASA needed 
common definitions for what constitutes the various aberrant events. Therefore, these definitions 
are provided for consideration in a future NASA Guidebook: 


Problem: An adverse situation, event, or condition that exists at a specific moment 

in time; any adverse event or condition that requires attention, resources, 
time, and/or effort to resolve. 


Nonconformance: Condition where an item has failed to comply with specified requirements. 

Anomaly: An unexpected event, hardware or software damage, a departure from 

established procedures or performance, or a deviation of system, 
subsystem, and/or hardware or software performance outside certified or 
approved design/performance specification limits 


Remedial Action: Action taken to make the aberrant article or material acceptable for use. 


Recurrence Control: Preventive measures to significantly reduce the likelihood, mitigate the 
adverse effects or effectively eliminate the possibility of recurrence of a 
aberrant condition 


Corrective Action: Correction, replacement, repair, or authorized disposition of noncompliant 

items/conditions, implementation of preventive measures to eliminate the 
causes of noncompliance, and validation that implemented preventive 
measures have effectively eliminated recurrence of the aberrant condition 
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11.0 List of Acronyms 


ARC 

Ames Research Center 

CIL 

Critical Items List 

FMEA 

Failure Mode and Effects Analysis 

GIDEP 

Govemment/Industry Data Exchange Program 

GSFC 

Goddard Space Flight Center 

IRIS 

Incident Reporting Information System 

IT 

Information Technology 

JPL 

Jet Propulsion Laboratory 

JSC 

Johnson Space Center 

KSC 

Kennedy Space Center 

LaRC 

Langley Research Center 

LDAP 

Lightweight Directory Access Protocol 

MSFC 

Marshall Space Flight Center 

NASA 

National Aeronautics and Space Administration 

NESC 

NASA Engineering and Safety Center 

NISE 

National Institute for Science Education 

RDF 

Raw Data File 

URL 

Uniform Resource Locator 
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13.0 Minority Report (dissenting opinions) 

Not applicable. 
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VOLUME II: APPENDICES 

Appendix A. Recommended Data Elements and Taxonomies for 
Problem Reporting 
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Table 1: Data Elements 


Field 

Share 

Source 

Definition 

EXPECTED 
PROBLEM 
REPORT 
PROCESSING 
PHASE(S) FOR 
DATA FIELD 
POPULATION: 
1=lnitiation, 
2=Analysis, 
3=Closeout 

Problem Identifier 

* 

Computer 

generated 

Computer-generated unique identification 
number, based on some predetermined 
scheme 

1 

Problem Title 

* 

Limited 
length text 
string 

Short description of the problem (100 -120 
characters), indicating the what, when, 
where 

1 

Problem 

Description 

* 

Large text 
string 

Detailed description of the problem - 
"prescription" for what would be 
information to be included is provided 

1 

Problem Type 


Pick list 

Categorization of the type of problem 

1 

Initiator POC 


People 

Taxonomy 

Name, organization, email, telephone, & 
role of person who initiated the problem 
report 

1 

Problem 

Occurrence Date 

* 

Formatted 

String 

Date that problem occurred 

1 

Problem 

Occurrence Time 


Formatted 

String 

Time that problem occurred 

1 

Occurrence 

Location 

*New 

Text string 

Geographical or orbital location of the 
anomalous item when problem occurred 

1 

Prevailing 

Conditions 

*New 

Text string 

Environment in which the anomalous item 
existed when the problem occurred 

1 

Detection Date 


Formatted 

String 

Date when problem was detected 

1 

Detection Time 


Formatted 

String 

Time when problem was detected 

1 

Detection Location 


Text string 

Geographical or orbital location of the 
anomalous item when problem was 
detected 

1 

Detecting During 


Text string 

Description of the activity that led to 
detection of the problem, e.g., analysis, 
text, maintenance 

1 

Program 


Taxonomy 

Program name (program attributes defined 
in NPR7120.5C) 

1 
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Field 

Share 

Source 

Definition 

EXPECTED 
PROBLEM 
REPORT 
PROCESSING 
PHASE(S) FOR 
DATA FIELD 
POPULATION: 
1=lnitiation, 
2=Analysis, 
3=Closeout 

Project 

* (1 of 
2) 

Taxonomy 

Project name (project attributes defined in 
NPR7120.5C) 

1 

Mission Name 

Taxonomy 

Mission name within project, e.g., STS 
114, GOES-N 

1 

Mission Type 


Pick list 

Type of mission 

1 

Lifecycle Phase 

* 

Pick list 

Phase of mission when problem occurred 

1 

Vehicle/ 

Spacecraft Type 

* 

Pick list 

This is either the spacecraft type or a 
particular vehicle name 

1 

Payload/Instrument 

Name 


Taxonomy 

Name given to the element within the 
mission that is gathering the science data 

1 

Payload/Instrument 

Type 


Taxonomy 

Type of instrument or payload 

1 

Immediate 

Response 


Text string 

Description of initial actions that were 
taken to respond to the problem as soon 
as it was discovered; e.g., remove-replace, 
securing 

1 

Failure 

Mode/Symptoms 

* 

Pick list 

The manner in which an item can or 
actually failed to perform its required 
function within specified limits, under 
specified conditions, for a specified 
duration; an actual component failure/error 
mis-performance that was an initial event 
in occurrence of an anomaly; includes 
indications that a problem/issue exists; a 
way that a component failure, fault, or 
unsatisfactory condition becomes 
apparent; physical characteristics 
displayed by anomalous performance of a 
component or assembly 

1-2 

Defect 

Characteristics 

*New 

Example 

list 

A fault/flaw/discrepancy/nonconformance 
in a component or process that causes 
discrepant performance of the component 
or assembly involved 

2 

Anomalous Item 
State 

*New 

Example 

list 

Identification of the state or configuration of 
the anomalous item when the problem 
occurred 

1-2 

Material Involved 


Example 

list 

Identification of materials related to the 
anomalous item; e.g., gases, liquids 

1-2 
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Field 

Share 

Source 

Definition 

EXPECTED 
PROBLEM 
REPORT 
PROCESSING 
PHASE(S) FOR 
DATA FIELD 
POPULATION: 
1=lnitiation, 
2=Analysis, 
3=Closeout 

Recurrence 
Control Required? 


Yes/No 

Yes indicated that this problem has the 
potential to occur on other missions or 
systems - it's a generic issue 

1-2 

System 

* 

as 

avail 

Generic 

subsystem 

Hierarchical identification for various levels 
to pin-point where the anomalous item fits 
within the system (various levels are 
defined in the NASA SE Handbook) - Item 
history includes use time & cycles, design 
use time & cycles, longest observed use 
time & cycles 

1-2 

Subsystem 

Generic 

subsystem 

1-2 

Assembly Level 

Example 

list 

1-2 

Assembly/ 

Component/Part 

Name 

Many to 
one 

relationship 
between 
these 
levels and 
the higher 
levels 
beginning 
with 

Subsystem 

1-2 

Assembly/ 

Component/Part 

Number 

1-2 

Assembly/ 
Component/Part 
Serial Number 

1-2 

Assembly/ 

Component/Part 

Manufacturer 

1-2 

Assembly/ 

Component/Part 

Integrator 

1-2 

Item History 

1-2 

Hardware 
Criticality Code 


Pick list 
(auto fill) 

Criticality code assigned to particular 
hardware based on FMEA and CIL 

1-2 

Criticality Code 

* 

Pick list 

Assessment of the severity of the problem 
based on FMEA and CIL for the assembly 
level ... this is the functional criticality level 

1-2 

Item Disposition 


Text string 

Description of what was done with the 
anomalous item; e.g., repair, return to 
vendor 

1-2 

Mishap Report? 


Yes/No 

Did this problem result in a formal mishap 
report due to damage of equipment or 
personal injury? 

1-2-3 
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Field 

Share 

Source 

Definition 

EXPECTED 
PROBLEM 
REPORT 
PROCESSING 
PHASE(S) FOR 
DATA FIELD 
POPULATION: 
1=lnitiation, 
2=Analysis, 
3=Closeout 

Adverse Program 
Impact 

*New 

Pick list 

Identification of adverse effects resulting 
from the problem; e.g., schedule delay, 
missed test date 

1-2-3 

Analysis POC 

* 

People 

Taxonomy 

Name, organization, email, telephone, & 
role of person who has been assigned to 
analyze the problem 

1-2 

Previous 

occurrence? 


Yes/No 

Has this or a similar problem happened 
before in this mission or others? 

1-2 

Related Problems 


Text string 

Description of related problems including 
the problem identifiers and how this 
problem is different or similar to those; this 
could also include descriptions of noticed 
irregularities than did not generate formal 
problem records 

1-2 

Waiver/ Deviation? 


Yes/No 

Has a waiver or deviation been issued for 
this type problem before? 

1-2 

Waiver/Deviation 

Info 


Text string 

Description of applicable waivers/deviation 
documentation 

1-2 

Material Review 
Board? 


Yes/No 

Does this problem need to be referred to 
the Materials Review Board? 

1-2 

Process Escape? 

* 

Yes/No 

Should this problem have been prevented 
by some established process? 

1-2 

Process 

Description 

*New 

Text string 

Description of the process that should have 
prevented this problem including 
identification of the process & the 
circumstances associated with missing the 
problem 

1-2 

Requirement 

Violation? 


Yes/No 

Was this problem in violation of the 
functionality of the 

system/subsystem/assembly/component/p 

art? 

1-2 

Requirement 

Violation 

Description 


Text string 

Description of the requirement that was 
violated & the mitigating circumstances 

1-2 

Usage Constraints 


Text string 

Description of the constraints that were 
immediately applied as a result of this 
problem until the problem is resolved 

1-2 

Applicable 

Documents 


Example 

list 

Identification of references/documents that 
are applicable to this problem; e.g., CIL, 
HAZ, GIDEP, FMEA 

2 
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Field 

Share 

Source 

Definition 

EXPECTED 
PROBLEM 
REPORT 
PROCESSING 
PHASE(S) FOR 
DATA FIELD 
POPULATION: 
1=lnitiation, 
2=Analysis, 
3=Closeout 

Root Cause 

Analysis 

Techniques 


Example 

list 

Identification of the root cause analysis 
techniques and/or tools that were used in 
the analysis; e.g., fault tree, Relex, Reason 

2 

Contributing Factor 
Category(s) 

*New 

Pick list 

Classification of the contributing factor(s) 
for the problem 

2 

Contributing 

Factors 

* 

Text string 

Description of the contributing factors to 
this problem (a contributing factor is an 
event or condition that may have 
contributed to the occurrence of an 
undesired outcome, but if eliminated or 
modified, would not by itself have 
prevented the occurrence) - "prescription" 
for what would be information to be 
included is provided 

2 

Probable Cause(s) 

* 

Text string 

Description of the probable cause(s) for 
this problem (a probable cause is a factor 
that is believed to have contributed to or 
created the undesired outcome)- 
"prescription" for what would be 
information to be included is provided - 
either there is a root cause or a probable 
cause (not both) 

2 

Root Cause 
Category(s) 

* 

Pick list 

Classification of the root cause(s) for the 
problem 

2 

Root Cause(s) 

* 

Text string 

Description of the root cause(s) for this 
problem (a root cause is one of multiple 
factors (events, conditions, organizational 
factors, etc.) that contributed to or created 
the proximate cause and subsequent 
undesired outcome, and if eliminated or 
modified, would have prevented the 
undesired outcome)- "prescription" for what 
would be information to be included is 
provided - either there is a root cause or a 
probable cause (not both) 

2 

Proximate Cause 
Category(s) 

*New 

Pick list 

Classification of the proximate cause(s) for 
the problem 

2 
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Field 

Share 

Source 

Definition 

EXPECTED 
PROBLEM 
REPORT 
PROCESSING 
PHASE(S) FOR 
DATA FIELD 
POPULATION: 
1=lnitiation, 
2=Analysis, 
3=Closeout 

Proximate 

Cause(s) 

*New 

Text string 

Description of the most immediate 
proximate cause(s) for this problem (a 
proximate cause is one of multiple factors 
(events or conditions) that occurred, 
including any condition(s) that existed 
immediately before the undesired outcome, 
directly resulted in its occurrence, and, if 
eliminated or modified, would have 
prevented the undesired outcome)- 
"prescription" for what would be 
information to be included is provided 

2 

Expected Date 
Root Cause(s) 


Formatted 

String 

Expected date for determination of root 
cause(s) 

2 

Actual Date Root 
Cause(s) 


Formatted 

String 

Actual date for determination of root 
cause(s) 

2 

Potential Future 
Impact? 


Yes/No 

Are there potential ripple effects of this 
problem within this mission or other 
missions? 

1-2 

Potential Future 
Impact Description 


Text string 

Description of the potential ripple effects of 
this problem within this mission or other 
missions, including dependencies among 
components, existence of common 
components, effectivity 

1-2 

Resolution POC 


People 

Taxonomy 

Name, organization, email, & telephone for 
person responsible for resolution 
development 

2 

Implementation 

POC 


People 

Taxonomy 

Name, organization, email, & telephone for 
person responsible for implementation of 
the problem resolution (remedial and/or 
corrective) 

2 

Expected Date 

Solution 

Development 


Formatted 

String 

Expected date for development of solution 
to resolve problem (remedial and/or 
corrective) 

2 

Expected Date 

Solution 

Implementation 


Formatted 

String 

Expected date for implementation of 
solution to resolve problem (remedial 
and/or corrective) 

2 

Interim Resolution 


Text string 

Description of the problem resolution 
including plan of action & rationale 

2 
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Field 

Share 

Source 

Definition 

EXPECTED 
PROBLEM 
REPORT 
PROCESSING 
PHASE(S) FOR 
DATA FIELD 
POPULATION: 
1=lnitiation, 
2=Analysis, 
3=Closeout 

Interim Approval 
Responsibility 


People 

Taxonomy 

Name, organization, email, & telephone for 
person responsible for approval of interim 
resolution 

2 

Remedial Action 


Text string 

Description of resolution to correct the 
problem in its current occurrence - 
"prescription" for what would be 
information to be included is provided 

2 

Corrective Action 

* 

Text string 

Description of final resolution to prevent 
reoccurrence of this problem or to minimize 
its impact - a systemic fix - "prescription" 
for what would be information to be 
included is provided 

2 

Residual Risk? 


Yes/No 

Is there remaining risk in using this 
item/system after implementation of final 
resolution (after corrective action)? 

2-3 

Residual Risk 
Description 


Text string 

Description of the remaining risk factors 
(after corrective action) in using this 
item/system after implementation of final 
resolution 

2-3 

Impacted 

Documents? 


Yes/No 

Are any documents invalidated or revisions 
required as a result of this problem? 

2-3 

Impacted 

Document 

Description 


Text string 

Description of documents that require 
revision as a result of this problem 
including title, reference number, schedule 
for revision, etc 

2-3 

Resolution 

Approver(s) 


People 

Taxonomy 

Name, role, date for approver(s) of final 
resolution that corrects the problem 

2-3 

Concurrence(s) 


People 

Taxonomy 

Name, role, date for people that need to 
concur with the resolution for this problem, 
e.g., ITA, review boards, project manager 

2-3 

Dissenting 

*New 

Text string 

Description of reasons for non-concurrence 
with the problem resolution, including 
name, date, role of person dissenting 

2-3 

Problem Closeout 
Summary 

* 

Text string 

Description of the problem resolution 
implementation including results 

2-3 

Actual Date 

Solution 

Development 


Formatted 

String 

Actual date(s) for development of problem 
solution 

2 
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Field 

Share 

Source 

Definition 

EXPECTED 
PROBLEM 
REPORT 
PROCESSING 
PHASE(S) FOR 
DATA FIELD 
POPULATION: 
1=lnitiation, 
2=Analysis, 
3=Closeout 

Actual Date 

Solution 

Implementation 


Formatted 

String 

Actual date(s) for implementation of 
problem solution 

2 

Configuration 

Change? 

*New 

Yes/No 

Was there a configuration change as a 
result of this problem? This would 
generate an automatic notification to key 
persons. 

2-3 

Follow-on Action? 


Yes/No 

Is follow-on action required as a result of 
this problem (other than configuration 
change, i.e. , procedural)? 

2-3 

Follow-on Action 
Description 


Text string 

Description of the follow-on actions 
assigned as a result of this problem 
including who is actionable 

2-3 

Lesson Learned? 


Yes/No 

Is there a lesson learned resulting from this 
problem? 

2-3 

Lesson Learned 
Description 


Text string 

Brief description of lesson learned from this 
process of identifying/working/resolving the 
problem, including link to lessons learned 
database item with more details 

2-3 

Notification? 


Yes/No 

Does the flight, ground crew, or others 
need to be notified of the problem? 

1-2-3 

Notification 

Identification 


Text string 

Identification of who needs to be notified as 
a result of this problem occurrence 

1-2-3 

Owner of 
anomalous item 


Text string 

Identification of who owns the 
hardware/software that experienced the 
problem 

1-2 

Problem Status 

* 

Pick list 

Identification of the current status of this 
problem, e.g., open, assigned, closed 

1-2-3 

Last Update Field 

* 

Formatted 

String 

Automatically filled by software when 
record saved 

1-2-3 (auto) 
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Table 2: Pick Lists 


NOTE: These are suggestions, individual projects may create other schemas 
or add additional values to these. Where these values are used, the definitions 

should be consistent. 


Field 

Potential Values 

Definitions of Values 

Problem Type 

Catastrophic failure 

loss of spacecraft and/or loss of crew 


Failure to meet primary 
objective(s) 

loss of ability to meet any primary or level 1 
mission requirement/objective 


Partial failure 

loss of ability to meet secondary or non-level 1 
mission requirement/objective, partial loss of 
system functionality 

can apply to any 
level such as 
system, subsystem, 
component, etc. 

Nonconformance or 
discrepancy 

system performance is outside specifications or 
requirements (e.g., parameter outside a 
specification limit), but no adverse impact to 
mission requirements/objectives 

Performance degradation 

adverse system performance trend (system 
performance degrading overtime), system 
operating outside control limits but within 
specification limits 



Unexpected/unexplained 
performance level 

as stated 


Other 

specify in a text field 

Mission Type 

Crewed (human) 



Uncrewed (robotic) 



Human/robotic 



Earth-observing 



Planetary 



Orbital 



Lander 



Solar System 



Deep Space 



Suborbital 



Other 
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Field 

Potential Values 

Definitions of Values 

Vehicle/Spacecraft 

Crewed Escape Vehicle 


Type 

(CEV) 

Crewed Launch Vehicle 
(CLV) 

Shuttle 

International Space Station 



satellite 

This field could be used as shown in the 


balloon 

rover 

probe 

launch vehicle 
lander 

sounding rocket 

aircraft 

other 

potential values area or could contain the 
vehicle name such as Endeavor, Discovery in 
the case of Shuttle 

Failure 

communication 

Individual projects need to go to a much lower 

Mode/Symptoms 

guidance & control 

level, this data item is intended to be a tiered 

and Proximate 

effect. For example, mechanical's 

Causes 

software 

next tier could be: buckled, corrosion, creep 


(plastic deformation), ductile deformation, 


electrical 

fatigue, fretting, galling & seizure, impact, 


mechanical 

radiation, rupture, spalling, wear. For 


structural 

example, electricals' next tier could be: 


material 

bonding defect, breakdown, contamination, 


propulsion 

cracking, diffusion, fatigue, hot carrier induced 


environment 

degradation, latch-up, mask defects, noise, 


overstress or incorrect current magnitude, 


contamination 

documentation 

optical 

thermal 

system interface 
system-human interface 
other 

punch-through, voiding. 
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Field 

Potential Values 

Definitions of Values 

Lifecycle Phase 

Manufacture 

terrestrial manufacture, testing, and evaluation 
of components and subsystems 


Assembly & Integration 

terrestrial assembly, integration, and testing of 
the overall system 


Launch Site Operations 

terrestrial launch site processing of any 
spacecraft (launch vehicles and payloads), 
design and operation of associated ground 
support systems, and launch control operations 


Flight Operations 

includes launch ascent, on-orbit operations, in- 
transit operations, landing operations, and 
associated mission control operations 


Surface Operations 

includes rover/robotic operations, surface crew 
operations, non-terrestrial surface 
manufacturing/resource production, non- 
terrestrial launch/landing site preparation and 
spacecraft processing, and associated mission 
control operations 


Landing Site Operations 
(for reusable/recoverable 
systems only) 

terrestrial post-landing and/or recovery 
operations 


Maintenance and 
Refurbishment (for reusable 
systems only) 

terrestrial maintenance and refurbishment of 
any reusable system, including reusable launch 
vehicles and reusable payloads/payload 
containers 

Root Cause 
Category (could be 
contributing factors 
also) 

Management 

includes causes resulting from organizational 
structure, oversight, resource allocation, 
planning, commitment, roles/responsibilities, 
control of work processes, leadership of 
organizational culture, organizational 
performance measurement, internal 
relationship management (i.e. , unions, 
employees), external relationship management 
(i.e., customers, suppliers, regulators) 


Policy 

includes causes resulting from policy 
documentation, clarity of policy, enforceability 
of policy, communication of policy, basis of 
policy 


Communication 

includes causes related to the timeliness, 
completeness, objectivity, and delivery of 
communications 
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Potential Values 

Definitions of Values 

Supervision 

includes causes associated with how (well) 
supervisors provide leadership, rule 
enforcement, task preparation, employee 
support, etc., and how well acceptable 
behavior, responsibility, authority, etc. is 
delineated to personnel 

Personnel 

includes causes related to the qualifications, 
motivations, quantity, experience, morale, 
physical factors/anthropometrics, emotional 
factors, accepted work practices, team 
composition/dynamics, team composition, team 
adaptability/flexibility, and perceived barriers of 
workers 

Training 

includes causes related to the timeliness, 
completeness, currency, appropriateness of 
training (including system training, task training, 
emergency training, safety awareness training, 
leadership, and team skills training) as well as 
whether certifications are required and 
maintained 

Work Environment 

includes causes related to the work facility, 
platforms, and work stations - ergonomics, 
cleanliness, organization, temperature, 
humidity, etc. 

Material Resources 

includes causes related to support equipment, 
tools, parts, shop aids (reliability, usability, 
availability, certification, cleanliness, etc.) and 
procurement, logistics, and material control 
processes/systems. 

External Environment 

includes causes concerned with the external 
conditions experienced by the engineered 
system such as weather, ice, radiation, etc. 

Task design & performance 

includes causes resulting from error/omission, 
attention/distraction, complexity/difficulty, 
inadequate directions, insufficient response 
times, infrequent/unique tasks, flawed decisions 
of humans performing tasks, and other 
cognitive factors 

Safety program 

includes causes associated with the attention 
and implementation of the safety program such 
as its adequacy, resources, follow-through, 
reviews, assistance provided 
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Definitions of Values 

Potential Values 

Information system 

includes causes citing the reliability, accuracy, 
completeness, availability of information as well 
as accessibility, operability of the information 
system 

Procedures 

includes causes related to the use or 
application of procedures, such as whether they 
are current, complete, accurate, 
understandable, consider human factors; 
whether they are implemented correctly; 
whether compliance is audited etc. 

Codes, standards, 
guidelines 

includes causes associated with the use and 
application of codes, standards, technical 
controls, and guidelines such as whether they 
are correctly identified, appropriate, available, 
accurate 

Requirements/specifications 

definitions 

includes causes citing issues with requirements 
or specification definitions such as whether they 
are complete, clear, traceable, free of conflicts, 
correctly flow-down/roll-up 

System Design 

includes issues related to the performance of 
system (flight systems, ground support 
systems, and facility systems) design such as 
risk identification and mitigation, modeling, 
analysis, testing, parametric trades, meeting 
requirements, defining margins, understanding 
uncertainties and assumptions 

Risk/hazard analysis 

includes issues citing the adequacy of risk 
modeling, tracking, and communication such as 
whether the management process is 
continuous, rigorous, timely, controlled, utilizes 
independent assessment 

Reviews 

includes causes associated with the 
performance of reviews such as whether they 
are independent, have appropriate expertise, 
are timely, use a corrective action system, have 
the correct quantity and scope 

Change control 

includes causes citing change control or 
management-whether it is thorough, uses 
appropriate configuration management 
techniques, is documented, requires new risk 
assessments 
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Definitions of Values 

Potential Values 

Quality control 

includes causes associated with quality 
assurance (QA) roles and responsibilities--are 
there sufficient, adequate resources? what are 
the requirements and are they met? is QA 
considered and continued throughout all 
relevant project phases? Were appropriate 
statistical methods used? 

Project Management 

includes causes associated with schedule 
pressure, schedule conflicts, budget controls, 
etc. 

Operational readiness 

includes causes related to the adequacy of 
verification/validation activities such as 
integrated system tests, analysis of as-builts, 
proof tests 

Maintenance 

includes causes citing maintenance activities, 
preparation and implementation: have risks, like 
collateral damage potential, been considered? 
are requirements implemented, enforced, 
doable? Can actions be performed reliably; 
have human factors issues been considered? 
how are problems, changes handled, 
documented? 

Inspection 

includes causes citing inspection activities, 
preparation and implementation: have risks, like 
collateral damage potential, been considered? 
are requirements implemented, enforced, 
doable? Can actions be performed reliably; 
have human factors issues been considered? 
how are problems, changes handled, 
documented? 

Anomaly resolution 

includes causes related to anomaly 
identification, analysis, and resolution: how are 
precursors identified? are warning signs 
heeded? what is the definition of anomalous? 
how are differences between predicted & actual 
behavior handled? what is the resolution 
process? was it followed? 

Amelioration 

includes causes related to control of events 
once an accident or incident occurs, such as 
prevention of chain of events, containment, 
hazard control, contingency planning, use of 
redundancy, use of personnel protection 
equipment 
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Field 

Potential Values 

Definitions of Values 

Criticality Code 

1 

Single failure that could result in death or loss 
of vehicle 


1R 

Redundant hardware items that could cause a 
criticality 1 event if all items fail 


IS 

Safety or hazard monitoring hardware items 
that could cause the system to fail to detect, 
combat, or operate when needed during a 
hazardous condition, potentially resulting in a 
criticality 1 event 


2 

Single failure that could result in severe and/or 
permanent injury, major property damage, or a 
loss of mission 


2R 

Redundant hardware items that could cause a 
criticality 2 event if all items fail 


3 

Single failure that could result in minor injury, 
minor property damage, a significant mission 
delay, or a mission degradation in which some 
mission goals not achieved 


4 

All other failures that result in unscheduled 
maintenance or repair 

Adverse Program 
Impact 

Programmatics 

Problem adversely affects programmatic issues 
(e.g., personnel, equipment, facilities) 


Technical 

Problem adversely affects the technical 
performance of the system 


Cost 

Problem increases the expected final cost or 
adversely affects the budget phasing 


Schedule 

Problem increases the expected length of time 
required to accomplish the task or mission 


Safety 

Problem creates a safety issue 

Problem Status 

Open-Initiated 

Problem initially entered into system, but no yet 
assigned to organization/individual for action 


Open-Assigned 

Problem assigned to organization &/or 
individual for action but no further actions taken 
to date 


Open-T roubleshooting 

Troubleshooting in work to identify and isolate 
the nonconformance 


Open-Isolated 

Problem isolated to assembly/component(s) but 
cause not yet determined 


Open-Cause Analysis 

Cause for problem determined, but corrective 
action not yet determined 
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Definitions of Values 

Potential Values 

Open-Corrective Action 
Development 

Correction plan of action to address cause has 
been determined but not yet implemented 

Open-Corrective Action 
Implementation 

Corrective action has been implemented 

Interim Disposition 

Temporary dispositioned for specific 
components/milestones/events, but not fully 
resolved for entire fleet 

Government Disposition 
Review 

Disposition in government approval/review 
cycle 

Follow-on Actions 

Problem resolved, but follow-on actions remain 
open (e.g., related documentation update) 

Closed-Action Taken 

Full closed, with actions implemented to 
address the cause(s) 

Closed-Explained 

Fully closed, with approved rationale that no 
actions are required to address the understood 
issue 

Closed- Unexplained 

Cause not fully understood, but closed by 
addressing the issue as best possible through 
mitigation and/or resolution of 
probable/possible cause(s) 

Hold 

Issue approved for leaving unresolved and not 
being actively addressed at present time 

Void 

Problem erroneously entered and should not be 
present (e.g., duplicate, mistaken data entry, 
non-problem no nonconformance) - DATA 
RECORDS SHOULD NOT BE REMOVED 
FROM THE SYSTEM 
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Table 3: Examples 


NOTE: These are examples, individual projects should create pick lists or 
links wherever possible to represent their situation. These data fields were 
included here because the team did not believe that even a high level set of 
values would be consistent between projects. 

Field Potential Values 

Definitions of Values 

Anomalous Calibration mode 

Item State Safe mode 

Open 
Closed 

Depends on the specific characteristics of 
the mission components. 

Assembly Line-Replaceable 

Level unit (LRU) 

Shop- 

Replaceable Unit 
(SRU) 

Crew- 

Replaceable Unit 
(CRU) 

Turbine Blade 
(NCA) 
Turbopump 
(LRU) 

Component 

Assembly 

Part 

Depends on the specific characteristics of 
the mission components. Could put in 
name for that level of the actual assembly. 

Defect Miswired 

Characteristics Abraded 

Dinged 
Part omitted 
Worn 

Short-circuited 

Delaminated 

These are definitely not an exhaustive list. 

Material Consumables Hypergolic fuel 

Involved Hypergolic 

oxidizer 

Air 

Hydrogen 
Oxygen 
Purge gases 
Potable water 
Food 
Other 

Serviceable Hydraulic 

Fluids fluids 

This envisioned to be a multi-tiered field as 
shown. 

Brake fluids 
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Serviceable 

Materials 

Other 

Pyros 

Other 


Applicable CIL xxx 



Documents FMEAxxxx 

Lessons Learned 

XX 


Text field with links to areas where 
applicable documents can be reached. 

Root Cause RELEX 

Analysis Fault Tree 

Techniques 


Text field with links to areas where 


applicable documents can be reached. 
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Table 4: 

Taxonomies 

These fields should be linked into NASA's formal taxonomies to auto-fill 
where possible. 

Field 

Taxonomy 

Comments 

Imitator POC 


NASA's classification project (NISE), 

POPS2: People, Organizations, Projects, Skills 

Program 


NASA taxonomy 

htto://lurch.ha.nasa.aov/2005/1 1/21/ooos.owl 

Project 


NASA taxonomy 

htto://lurch.ha.nasa.aov/2005/1 1/21/ooos.owl 

Mission Name 

STS-114 

GOES-N 

NASA Taxonomy does have mission names 
specified. 

Payload/Instrument Name 

MODIS 

NASA Taxonomy does have payload/instrument 
names specified. 


GIFTS 

Individual centers also have taxonomies. 

Payload/Instrument Type 

telescope 

NASA taxonomy 


spectrometer 

imaging 

lidar 

http://lurch.hq.nasa.qov/2005/1 1/21/pops.owl 

Analysis POC 


NASA's classification project (NISE), 

POPS2: People, Organizations, Projects, Skills 

Resolution POC 


NASA's classification project (NISE), 

POPS2: People, Organizations, Projects, Skills 

Implementation POC 


NASA's classification project (NISE), 

POPS2: People, Organizations, Projects, Skills 

Interim Approval 


NASA's classification project (NISE), 

Responsibility 


POPS2: People, Organizations, Projects, Skills 

Resolution Approver 


NASA's classification project (NISE), 

POPS2: People, Organizations, Projects, Skills 

Concurrence 


NASA's classification project (NISE), 

POPS2: People, Organizations, Projects, Skills 
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Table 5: Subsystem Choices 

Definition for System: 

Type of element to which the anomalous item applies. 
Provides context for the subsystems in the Subsystems 
Field. 

System Name 

Definition 

Robotic Spacecraft 

Any non-crewed flight system that is used to achieve a set of 
mission objectives. Includes items such as orbiting satellites (e.g., 
Terra, TDRSS and MRO), space telescopes (e.g., HST), rovers, 
landers, probes, and balloons. 

Crewed Spacecraft 

Vehicle which carries humans into space or supports them in 
space (e.g., Shuttle, ISS, and CEV). 

Instrument/Payload/Experiment 

The system or systems that are accomplishing a mission's 
objectives (e.g., taking science measurements or pictures). An 
instrument may be supported on a Spacecraft (e.g., MODIS 
instrument on the Terra spacecraft). It can also be the system that 
is being carried/supported by a Crewed Spacecraft or Launch 
Vehicle. 

Aircraft 

Missions that are carried out using an airplane or similar powered 
vehicle that remains in the atmosphere. 

Launch Vehicle 

Any type of rocket-based system that propels another vehicle from 
the ground (e.g., Earth, Moon, or Mars) into space. Also includes 
sounding rockets. 

Ground System 

An element that provides its support from the ground. 

Other/Unknown 

Not covered by other selections. 

Definition for Subsystem: 

The applicable subsystem associated with the specified 
System. Generally the subsystems are the first level 
breakdown in the hierarchy of elements that make up a 
System. 

Subsystem Name 

Definition 

Command & Data Handling 

Includes all non-GN&C flight hardware and software that support 
the handling, processing, and storage of data. Includes items such 
as the main flight computer, spacecraft bus, stored command 
processor, wiring harnesses, and Solid State Recorder. 

Guidance, Navigation & Control 

Supports attitude and orbit control of vehicle (e.g., reaction wheels, 
gyros, magnetic torquers, GPS receivers, etc. 

Communications (RF) 

Provides radio frequency communications among spacecraft and 
ground systems (voice and data). 

Mechanical - Structures 

Physical structures that comprise a vehicle, spacecraft, etc. 

Mechanical - Mechanisms 

Devices/hardware that enable motion, such as motors, wheels, 
and bearings. 
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Subsystem Name 

Definition 

Electrical Power 

Provides electrical power to other systems, and manages the power 
overall. Includes items such as solar panels, batteries, and electrical 
wiring. 

Propulsion 

Provides ability to perform maneuvers while in orbit (e.g., attitude or 
orbit adjustment), as well as ascent and descent operations. 

Environmental Control and Life 
Support 

Controls and monitors the environment in which a system resides. 
Provides life support to humans, plants, or animals in a vehicle. 
Controls items such as temperature, humidity, contamination, and air 
quality. 

Thermal 

Provides active and passive methods for controlling temperature during 
all phases of mission. 

Plumbing 

Provides for the flow of liquid (e.g., water and fuel) in a vehicle, 
spacecraft, etc. 

Pyrotechnic 

Uses devices or assemblies operated by solid propellants or explosive 
charges to perform separation and range safety, recovery, avionics bay 
fire suppression, emergency jettison, radar antenna rendezvous, 
docking (tunnel and radiator), and crew escape. 

Crew Equipment 

Pertains to all end items of installed, stowed and/or worn crew-related 
GFE and CFE required for crew members to accomplish a mission. 
These end items are those which the crew member utilizes, operates, 
or monitors, and are required to support crew member activities from 
ingress through egress. 

Range Safety 

Provides for safety at the range, to include items such as command 
receiver couplers, antennas, decoders, and control distributors. 

Ops Control Center 

Provides commanding and telemetry processing within the ground 
system. 

Ground Station 

Ground element that provides voice and data communications between 
the ground system and one or more space systems. 

Networks 

All ground data networks (LANs/WANs) and voice communications 

Front-End Processing 

Ground system frame/packet processing, line outage recording, data 
replay, data store and forward, etc. 

Data Processing/Distribution 

Ground data processing of science and housekeeping data (Level 0-4), 
distribution of data to customers, data archival, etc. 

Software 

Provides a variety of flight and ground support functions, across many 
Systems and Subsystems. 

Other/Unknown 

Not covered by other selections. 
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Table 6: Formatted Strings 


Field 

Formatting 

Problem Occurrence Date 

mm/dd/yyyy 

Problem Occurrence Time 

hh:mm:ss 

Detection Date 

mm/dd/yyyy 

Detection Time 

hh:mm:ss 

Expected Date Root Cause 

mm/dd/yyyy 

Actual Date Root Cause 

mm/dd/yyyy 

Expected Date Solution Development 

mm/dd/yyyy 

Expected Date Solution 
Implementation 

mm/dd/yyyy 

Actual Date Solution Development 

mm/dd/yyyy 

Actual Date Solution Implementation 

mm/dd/yyyy 

Last Update Field 

mm/dd/yyyy; 

hh:mm:ss 

Note: All times would be expected to be in local time. 
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Appendix B. Characteristics of a Good Taxonomy for 
Problem Reporting 

A White Paper for the NESC Taxonomy Working Group 

Jayne Dutra/JPL and Tim Barth/KSC 


Introduction 

Effective problem reporting systems include functions beyond data reporting and collection. The 
overall architecture of a problem reporting system can be described by five major subsystems: 
the data collection system, the data analysis system, information reporting system, feedback 
system, and management system. The effectiveness of taxonomy design impacts all five 
subsystems and the overall performance (or value) of the problem reporting system. 

The goal of a data system is to produce information that provides value to the users of that 
information. The users should gain information and insights that enable them to improve their 
decision-making. Taxonomies can act as a conceptual brokering layer so that data between 
systems can be aggregated by categories that are relevant to the user. The outcomes of improved 
decision-making capabilities may include improved safety, cost, and/or schedule performance. 

This white paper explores the characteristics of good taxonomies for problem reporting by 
addressing the following three questions: 

• What is a taxonomy? 

• What is a good taxonomy? 

• How do you know? 

Taxonomies within information systems will change over the lifecycle of the system, so they 
should be considered “living” documents. However, since the impact of a taxonomy on the 
overall system is extensive, early investments in taxonomy design usually yield high returns. 

What is a Taxonomy? 

Most problem reporting systems include multiple taxonomies. A taxonomy is a “structure that provides a 
way of classifying things (such as living organisms, products, and books) into a series of hierarchical 
groups to make them easier to identify, study, and locate” [Bruno and Richmond, 2003]. The terms 
taxonomy, hierarchy, classification, faceted taxonomies, and ontology have overlapping and evolving 
meanings. These terms are discussed in the following paragraphs. 

The traditional definition of taxonomy is “the study of the general principles of scientific classification, 
especially orderly classification of plants and animals according to their presumed natural relationships” 
[Merriam- Webster, 2003]. A more general description of taxonomy is “the science of classification 
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according to a pre-determined system, with the resulting catalog used to provide a conceptual framework 
for discussion, analysis, and information retrieval. In theory, the development of a good taxonomy takes 
into account the importance of separating elements of a group (taxon) into subgroups (taxa) that are 
mutually exclusive, unambiguous, and taken together, include all possibilities. In practice, a good 
taxonomy should be simple, easy to remember, and easy to use. 

Another description of taxonomy is given by “structures that provide a way of classifying things - living 
organisms, products, books - into a series of hierarchical groups to make them easier to identify, study, 
and locate” [Bruno and Richmond, 2003]. Taxonomies are said to be made up of controlled vocabulary 
terms. Sometimes an information architecture will contain a namespace where controlled vocabulary 
authority (CVA's) files are kept as the "gold source" for certain naming lists. The goal of the NASA 
Taxonomy is to provide a gold source for NASA Project and Mission names, for example. Taxonomies 
change and grow as organizations, technologies and processes change and grow. 

Systematics is the “science of classification or a system of classification” and its meaning is 
closely related to taxonomy [Merriam- Webster, 2003]. Classification is “the process of 
classifying or a systematic arrangement in groups or categories according to established criteria” 
and hierarchy in this context is a “graded or ranked series” [Merriam-Webster, 2003]. 
Hierarchies consist of multiple layers or levels. 

Taxonomies contain predefined hierarchies of descriptors for marking content chu nk s. Faceted 
Taxonomies are composed of discrete branches which are also kn own as facets. Facets are made 
up of hierarchically organized lists of attribute values for use in consistently labeling content 
components with repeatable attributes. In the past, librarians could only place a book in one 
location on a library shelf. Today, our electronic technology allows us the opportunity to present 
information from multiple viewpoints maximizing the probability of discovery of relevant 
information by the end user. Facets allow taxonomies to be multi-dimensional, which 
accommodates a wider range of content types. Taxonomies that are designed to be modular and 
extensible will be the most robust. 

Applying values with the semantic consistency of a taxonomy enables search across multiple 
heterogeneous systems. If a taxonomy is meant to act as a conceptual brokering layer for data 
reconciliation, term development may be desirable at a broader level. The breadth of terms 
allows for many mapping points from various systems. Thus what one system calls a "Failure 
Report" may be labeled a "Problem Log" by another system, but both may fall under the broader 
category of "Quality Control Reports" in a brokering taxonomy. In addition, it is very possible 
that clues to failures may reside in repositories and systems meant to house unstructured 
information as well as databases. If controlled vocabulary terms are used consistently in data 
architectures and as values for metadata fields, then information from both sources may be 
correlated to give a more complete picture than the more one dimensional view that a single 
source can represent. 
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Figure 1 - Yahoo search results for "trial." From: Taxonomy at a glance. 

http://enterprise.yahoo.com/portal/services/taxonomy/taxonomyl.pdf . Last checked Jan. 6, 

2003 

In the last decade, ontology has become a fashionable term inside the knowledge engineering 
community, and many software tools are being developed to build ontologies that help 
organization of information, usually on the internet [Corcho, Fernandez -Lopez, Gomez-Perez, 
2003]. In the Semantic Web world, an ontology is most often defined as a representation of a 
body of knowledge defined in such a way as to be consumable by systems as well as humans. 
Ontologies will usually incorporate some form of controlled vocabularies or taxonomies with the 
addition of conceptual relationships built into the schema to give it more richness and depth. If a 
taxonomy is built with thesauri relationships inherent in the hierarchical structure (i.e., Broader 
Than, Narrower Than, Related To, etc.), then it can also be considered to be an ontology and 
rendered in an XML logic language like OWL using predicates that explicitly express the 
relationships. 

Ontologies can be used as interchange formats, enabling mediation between systems in a Web 
Services model. When implemented with controlled vocabularies and taxonomic underpinnings, 
ontologies enhance reusability, search results, reliability, and knowledge acquisition. Ontologies 
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and topic maps can allow us to catalogue what we know and what we don’t know, helping to 
shape our future research efforts as an Agency. 

“ Ontology (with an upper-case “O”) is the branch of philosophy that studies the nature of 
existence and the structure of reality. Ontology (with a lower-case “o”) investigates the 
categories of things that exist or may exist in a particular domain and produces a catalog that 
details the types of things - and the relations between those types - that are relevant for that 
domain” [Jacob, 2003]. 

What is a Good Taxonomy? 

Developing a taxonomy is relatively easy, but developing a good taxonomy is usually not so 
easy. Think about how you organize fdes on your computer. By creating folders, sub-folders, 
and moving specific files into those folders, you are creating your own personal taxonomy for 
organizing information. Taxonomy design and development is usually based on a combination 
of science, individual and organizational experience, intuition, and art. Objective and subjective 
factors influence the content and structure of a taxonomy. How often do you need to search for 
files on your computer, even though you organized the files using a taxonomy of folders that you 
designed? Good taxonomies need to be designed with many current and projected users in mind. 


Best Practices in Taxonomy Development 

The following terms describe current industry best practices in the library science and 
information architecture communities for the development of robust taxonomies. 

Hierarchical Granularity . The taxonomy is designed to provide as much depth or hierarchical 
granularity in the classification as the content requires. Because NASA's content includes highly 
technical subject matter, this allows authors to tag their material with precision, which also 
enables better search mechanisms and trending tools. 

Polyhierarchy . The taxonomy allows the same concept to reappear multiple times in the scheme. 
Because the same concept can then have multiple parents, navigational pathways are built in that 
facilitate a search from numerous and different approach points. Instead of knowing exactly the 
right tenn to search on, a user can come from his or her own individual viewpoint and still locate 
the pertinent information. 

Mapping Aliases . To add more richness to the labeling scheme, abbreviations and alternate terms 
are mapped to the taxonomy. By planning for acronyms and synonyms early on, the taxonomy 
becomes more accessible to users that possess a deeper grasp of any one topic area. 

Existing Standards . Make efforts to adopt categories for standard genre and document types in 
the Problem Reporting Types facet of the taxonomy so that users can start with a common 
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understanding of what document frameworks they might be looking for. Re-use other 
engineering standards that the Agency might have in place for spacecraft systems and sub- 
systems. 

One perspective on the key characteristics of a good taxonomy is that “ideal taxonomies": 

• Are hierarchically structured, 

• Have classes (categories) that clearly describe different aspects of the data 

• Cover all subjects of interest, 

• Are designed at the level of detail needed by database users, and 

• Employ defined, accessible, operationally meaningful terms that can be consistently 
applied by database coders. [Rosenthal, 1998] 


How Do You Know? 

How do you know if a taxonomy is “good?” The short answer is that the taxonomy fulfills its 
intended function. However, there are varying degrees of “goodness,” so the characteristics of a 
good taxonomy should be measurable. Shappell and Wiegmann [2003] define four factors 
affecting the validity of taxonomy: comprehensiveness, reliability, diagnosticity, and usability. 
Flexibility was identified as an additional factor. An example framework for taxonomy 
measures/indicators based on these factors is listed below: 

Comprehensiveness 

• Breadth - % coverage of top-level hierarchy categories 

• Completeness (Depth) - % coverage of lower-level contributing factor categories 

Usability 

• Subjective evaluation of reporter/coder/user to cover ease of use, intuitive structure 

• Average time to complete data entry/coding after all information is collected/understood 

Diagnosticity (event-specific and systemic) 

• Demonstrate additional diagnostic capability/insights 

• Number and effectiveness of corrective/preventive actions taken based on analysis 
results/recommendations 

• Event specific - reduction in number and severity of recurrences 

• Systemic - % improvement in performance trend over time 

Reliability 

• % same categories selected for one event (repeatability) - multiple reporters with similar 
training and experience, all with the same information 
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Flexibility 

• % of use anticipated and unanticipated use case scenarios supported 

• # of different organizations/domains able to use the taxonomy 

• Presence of lower-level categories to collect additional information on most frequently 
occurring categories 

Since some of the taxonomy measures are competing, taxonomy designers (like 
hardware/software system designers) must make tradeoffs. For example, increasing taxonomy 
comprehensiveness may decrease taxonomy usability for data reporters. 

Validation 

The best way to validate a taxonomy is by running a catalogue against the terms and then 
examining the coverage profile of the material. If there are many information objects clustered in 
one area of the taxonomy, the term structure may need more granularity. If there are areas of the 
taxonomy that are underutilized, then perhaps some term pruning needs to be done. 

It is important to look at the use case scenarios for taxonomy applications. Some use cases for 
taxonomy applications are targeted content delivery (into a portal for example), identification of 
patterns in data mining, and data correlation between system objects that are not co-located. In 
addition, if the goal is to provide access to items in multiple heterogeneous repositories, then 
there should be a broader design in order to accommodate tenn variations. 

Summary 

The challenge is to optimize the taxonomy design by balancing these key performance 
characteristics so that the most significant attributes are revealed to the user in a mental model 
that is intuitive and logical. 
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Appendix C. Suggested Prescriptions for Values of Key Free Text Fields 

Remedial Action - Action taken to bring a specific failed unit to operational status or to 
eliminate an unsatisfactory condition on the specific unit; e.g., remove and replace, rework to 
print, material review board (MRB) disposition, etc. 

The problem resolution summary for a remedial action should include the following data: 

a. Problem clarification - Add sufficient narrative to describe the problem. 

b. Analysis/Investigation - Include the following information: 

• Troubleshooting results 

• Whether problem was repeatable 

• Conditions under which the problem occurred 

• Analysis results 

• Tests and/or efforts made to determine the problem cause 

• Summary of rationale which led to a most probable cause determination 

• Most probable cause 

c. Problem history - Include all kn own failures of this failure mode. Discuss general history, and 
checkout history of failed unit. Use counts if there are numerous failure reports. 

d. Effect on units in the field - Indicate whether the unit that failed, or a like unit, is on (or 
planned for) future flights. 

e. Last test able to detect the problem - Indicate what test will be conducted, and when in the 
vehicle flow (e.g., pre-launch, countdown, etc.). 

f. Methods of detecting in flight - Indicate how the crew (if manned) will be made aware of the 
problem, or what automatic system detects or corrects the problem. 

g. Mission effect - Indicate the criticality-related effect on the mission if the problem occurred in 
flight, or in the launch countdown. Indicate operational workaround procedures. 

h. Explanation rationale - Indicate why it is acceptable to fly or continue flying with no further 
corrective action. Include any test results, troubleshooting, and any other applicable information 
available that will further justify this rationale. If applicable, provide assurance that redundancy 
and/or alternate modes of operation do not negate each other. Discuss remedial action taken. If 
there is a valid reason why flight rationale is acceptable for only a limited number of flights, the 
limited effectivity and acceptable rationale shall be included for the limited flight(s). 

i. Corrective/remedial action for subsequent vehicles/hardware - This item is not applicable if the 
explanation is for all vehicles/hardware. If the problem is closed for subsequent 
vehicles/hardware, indicate the documentation (configuration control board directive, 
engineering order number, etc.) that authorized the corrective action, and relate it to the 
vehicles/hardware affected. 

Corrective Action - Action approved by the appropriate Government authority to correct a 
problem cause which includes, for example, one or more of the following dispositions: 

1 . Design change 
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2. Manufacturing method/procedure/process change 

3. Facility/test equipment change 

4. Test or operating procedure change 

5. Training or certification of personnel 

6. Maintenance procedure change 

7. Limit time or cycle of component 

8. Flandling or shipping change. 

NOTE: In addition to the above, a change in quality assurance inspection requirements may be 
needed. 

Examples of problems that could lead to a corrective action: 

a. Significant acceptance test procedure (ATP) or pre-ATP failures 

b. Generic problem affecting flight hardware 

c. Problem cannot be accommodated by existing flight rules/crew procedures, basic subsystem 
redundancy, or has other implications that present a safety of flight issue, i.e., problem is a 
constraint to flight 

d. Other significant events 

When corrective action is necessary, further reviews of the problem should be conducted, to 
determine if any additional data are required for assessment. Any additional procedural changes 
should be identified (e.g., Operations and Maintenance Requirements and Specifications 
Document or Flight Data File), as well as any hardware inspection or change out, or any other 
rationale to allow constraint removal. If no actions are identified, the constraint remains in effect 
until the problem is corrected or until additional information (such as failure analysis results) is 
obtained that justifies removal of the constraint. 

The corrective action shall address all of the hardware, including those units already delivered 
and in the field. If previous corrective action provisions exist and did not prevent the recurrence 
of the problem, then those provisions shall not be acceptable for defining further corrective 
action provisions, and the original problem report and any related problem reports shall be 
reopened and readdressed. 

Problem Description - A good problem description should use standard, consistent terms; 
minimize use of abbreviations; and include as much of the following information as is applicable 

1 . The operation being performed when the anomalous condition occurred &/or was 
detected (i.e., What was going on?). Include a description of the configurations, i.e., 
switch position, valve state, pressure, temperature, etc. as well as the date and time of 
occurrence. 

2. Description of where the anomaly occurred on the vehicle and the items that were 
affected by the anomaly. At a minimum should include the system/subsystem name(s) 
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and any other more detailed component information that is known and considered 
relevant to understanding the problem. 

3. The organization(s)/personnel operating the anomalous item when the incident 
occurred/was detected (i.e., Who was working on the item?) 

4. Location where the anomaly occurred/was detected (i.e., This is the physical location or 
environment of the vehicle, such as in-transit to or orbiting Mars, at the manufacturer, in 
thermal vacuum, or on-orbit - where did it happen?). 

5. Any normal &/or unusual circumstances &/or parameter reading preceding or during 
occurrence of the anomaly (i.e., What early indications were there, if any, that something 
might be going wrong or might have influenced what happened?). 

6. A description of the detected symptoms that led to discovery of the anomaly. This should 
include a description of relevant parameter data, as applicable, (presented as “IS” and 
“SHOULD BE”, referencing the pertinent requirement document) and/or any abnormal 
values or conditions. Any relevant pictures/schematics that help describe the problem 
should be included. 

7. The immediate actions taken to respond to the anomalous condition (i.e., What was done 
to immediately respond?). 

8. Any potential damage/injury/abnormal conditions immediately resulting from the 
anomaly (i.e., What happened as a result of the anomaly?). 

General Text Description Fields - Other description fields (free text) such as root cause, 
probable cause, proximate cause, and contributing factors should contain a detailed description 
of the factor including how it resulted in (or contributed to) the undesired outcome, what 
analyses techniques were employed to determine the relationship between the factor and the 
undesired outcome as well as the times and conditions associated with the occurrence. 
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